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We reanalyze constraints on the mass spectrum of the chiral fourth generation fermions and the 
Higgs bosons for the standard model (SM4) and the two Higgs doublet model (THDM). We find 
that the Higgs mass in the SM4 should be larger than roughly the fourth generation up-type quark 
mass, while the light CP even Higgs mass in the THDM can be smaller. Various mass spectra of 
the fourth generation fermions and the Higgs bosons are allowed. The phenomenology of the fourth 
generation models is still rich. 
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I. INTRODUCTION 

Repetition of the generation structure of quarks and 
leptons is a great mystery in particle physics. Although 
three generation models arc widely accepted, the basic 
principle of the standard model (SM) allows the sequen- 
tial fourth generation (family) [1, 2]. Also, the elec- 
troweak precision data does not exclude completely ex- 
istence of the fourth family [3-5]. Since the LHC has a 
discovery potential for the fourth generation quarks at 
early stage [6] , we may explore this possibility more seri- 
ously. 

If the fourth generation exists, it is well-known that 
the condensate of the fourth generation quarks t' and b' 
can dynamically trigger the electroweak symmetry break- 
ing (EWSB) [7]. In such a scenario, multiple composite 
Higgs bosons can naturally emerge as the scalar bound 
states of t', b 1 and other heavy fermions such as the top 
quark t and the fourth family leptons r' and v' [8, 9]. 
When the composite Higgs bosons composed of t, r' and 
v 1 are too heavy and hence inaccessible at Tevatron and 
LHC, the effective theory at the TeV scale will be a two 
Higgs doublet model (THDM) [10]. Furthermore, if the 
extra Higgs bosons other than the SM-like Higgs are de- 
coupled [11], the effective theory of the THDM is reduced 
into the SM with the fourth family (SM4). 

In this paper, we study the SM4 and also a THDM 
with the fourth generation. We assume Dirac-type neu- 
trinos. Models with Majorana mass terms will be studied 
elsewhere. 

The yukawa couplings of the fourth generation have 
the Landau pole, so that the SM4 or the THDM are ap- 
plicable up to at most several tens TeV. In this sense, it 
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is natural to expect the existence of some strong dynam- 
ics such as topcolor models [12] behind the SM4/THDM. 
Nevertheless, we will not impose the compositeness con- 
dition [13, 14], because we are interested in a wider class 
of models rather than the Nambu- Jona-Lasino type one. 

We reanalyze the stability condition(s) of the Higgs 
potential for the SM4 and the THDM [15, 16], and also 
impose the tree level unitarity bounds on the yukawa [17] 
and Higgs quartic couplings [18-20]. We then find the 
cutoff A at which some new physics enters to evade 
the instability of the Higgs potential or the perturbativc 
description breaks down owing to appearance of some 
strong dynamics. The cutoff A should not be so small. 
Otherwise, the models are not self-contained at the TeV 
scale. Besides the theoretical restriction, we take into 
account the constraints on the oblique parameters [21]. 

By varying all masses of the fourth generation fermions 
and the Higgs boson(s) within a reasonable parameter 
space, we obtain a set of favorable mass spectra. Strong 
correlations among the masses of the fermions and the 
Higgs bosons are found. It turns out that the Higgs 
mass in the SM4 should be larger than roughly the t' 
mass, while the light CP even Higgs mass in the THDM 
can be smaller because the dynamics of the extra Higgs 
quartic couplings can stabilize the Higgs potential against 
the negative contributions of the yukawa couplings. An- 
other noticeable consequence is that the decay channel 
t 1 — > v' + W~ is allowed in a wide parameter space 
in both of the SM4 and the THDM. The decay chan- 
nel t' ->• V + w<*> is not necessarily excluded. As for 
the Higgs, a decay channel into a pair of the fourth gen- 
eration neutrinos is kinematically open in a certain pa- 
rameter region. Depending on such possibilities, more 
comprehensive studies should be required. 

The paper is organized as follows: In Sec. II, we ana- 
lyze the SM4. In Sec. Ill, the THDM is studied. Sec. IV 
is devoted for summary and discussions. We show the 
renormalization group equations (RGE's) for the SM4 
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and the THDM in Appendix A and B, respectively. 



II. SM4 



where /io is an arbitrary scale. The universal fcrmion 
mass 777,4 is defined by 7774 = yi{[J- = m,A)v j \pl. By def- 
inition of the Landau pole A. y , = A y ) = and 
then we obtain the relation between A y and 7774 as 



Let us study the SM4. 
with 



(1) 



yt<ii tR<t> + ybqL J bn<f> + w<i£'t' R 4> + yvq£'b' R ^ 



MS) 
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RV, 
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where <f> represents the Higgs doublet field, <j> is defined 
by 4> = iT~2<t>* , and and £W denote the 7-th family 
doublet of quarks and lcptons, respectively. We take into 
account the yukawa couplings of the third and fourth 
generations, and ignore other yukawa couplings as well 
as the neutrino masses other than v' . As explicitly shown 
in C y , we simply assumed the Dirac-type neutrinos. 

The RGE's for the yukawa and Higgs quartic couplings 
are well-known [22, 23]. We show a set of the RGE's for 
the gauge, yukawa and Higgs-quartic couplings at the 
one-loop approximation in Appendix A. 

We explore the cutoff scale A of the SM4 at which some 
new physics or nonperturbative dynamics emerges. The 
point is that the yukawa coupling has the Landau pole at 
a certain energy scale A y and only an intermediate mass 
range of the Higgs boson is allowed by the triviality and 
instability bounds. Before the full one-loop calculation, 
we schematically describe nature of the RGE's. 

Let us solve analytically the RGE's under the following 
crude approximation. 

The electroweak gauge couplings are negligible. Al- 
though the QCD coupling is not so small, it behaves like 
a constant in the energy scale 0(1-10) TeV. On the other 
hand, the yukawa couplings for the fourth generation run 
very quickly and diverge at the Landau pole. Thus we 
may ignore all of the gauge couplings at the zeroth ap- 
proximation. For simplicity, we may neglect yt and also 
assume that all of the fourth generation yukawa couplings 
are the same as 7/4, although it is unrealistic because ow- 
ing to a relatively heavy Higgs, the T-parameter con- 
straint requires appropriate mass differences of the fourth 
generation fermions, which will be taken into account in 
the full analysis of the one-loop RGE's. 

Under the above crude approximation, the RGE for 7/4 
is given by 



(16^ 2 )/j^-7/ 4 = Syl 
and the solution is immediately found as 

1 ■ In fi/ no 



yi(») vt (mo) tt : 



(3) 



(4) 



A, 



7774 e 
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where v (= 246 GeV) is the vacuum expectation value 
(VEV) of the Higgs. Numerically, it yields 



for 



A y = 8 (10) TeV, 3 (3) TeV, 2 (2) TeV, (6) 



777.4 = 300 GeV, 400 GeV, 500 GeV . (7) 



Compared with the full one-loop calculation (the values 
in the parentheses), the approximation works in fact. 

As for the Higgs sector, within the above crude ap- 
proximation, the RGE for A is 

(167r 2 )/jJj-A = 24A 2 + ^ 32t/|A - 16t/ 4 j 0(ji - m 4 ), (8) 

where we explicitly treated the fermion contributions to 
the /3-function in the 9- function, i.e., below the thresh- 
old of 7774, the theory is matched to the SM without the 
fourth generation. We can easily find that the following 
quantity is the RGE invariant: 



a(m) - 

A(m) - £_ 



Mm)) 



-2V7 



(/J > max(7774 , 777,00 )), (9) 



where m^o (= ^/2A(/i = m^v) is the mass of the physi- 
cal Higgs boson and also 



\( n \ — A M , _ -l±V7 
2/4 W 3 



(10) 



Note that when 77 > 0, A goes to infinity at the scale 
A\[= A y exp(— ir 2 ^ 1 /^)], while it does to zero at the 
scale Ainst[= A y exp(-7r 2 

when 77 < 0. 

For 777 ^0 < 7774, the RGE for A develops only by the 
A 2 -term in the region m^o < fj, < 7774, so that it does not 
encounter instability in this region. For m^o > 7774, we 
do not need to care about the above threshold effects. 
Then, in terms of 7774 and to^o, the scale Aj nst at which 
A(/i = Ajnst) = is given by 



A; 



7774 exp 



2m 2 



1-C1 



8i/n 



\1 + C: 



, (11) 
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FIG. 1: Scatter plots of M t > v.s. M^o (left) and M„; v.s. M^o (right). The data points are the same in both figures. We 
varied 256 GeV < M t > < 552 GeV, 255 GeV < My < 552 GeV, 100.8 GeV < M T > < 1.23 TeV, 90.3 GeV < M v > < 1.23 TeV, 
and 114 GeV < Mj,o < 873 GeV, without any prejudice. We took into account all of the 40 patterns of the mass spectrum 
of the fermions and the corresponding threshold effects. The red, blue, magenta and green points correspond to the cutoff A, 
2 TeV < A < 3 TeV, 3 TeV < A < 4 TeV, 4 TeV < A < 5 TeV and A > 5 TeV, respectively. Below the cutoff scale A, the 
Higgs potential is stable and the perturbation is applicable. The data points are within the 95% C.L. limit of the S and T 
parameters. In the inset of the left figure, we showed the (S, T)-contour and the data points which we used for the scatter plot. 



with 



a/7 + 1 



3 m 



ttt rn — — 



<2 

for m^o < m<4, and 
Ainst = to 4 exp 



4tT 2 V* 7714 

y/7-1 



3 m- 



(12) 
(13) 



47T 2 U 2 m,4 



2 2 / 1 (v^+H ^ 



1 



i7r 2 v 2 l-<pO 



with 



t ao = ln ■ 



A, 



In 



2 2 
?Ti4 V 7T 



?7i0o 2m| ' 



(14) 



(15) 



for 77i0o > 7774. Similarly, the Landau pole A\ for A, i.e., 
A(/x = Aa) = 00, is given by 



with 



Aa = 777^,0 exp 



v{i-(i-Ca)^} 
2V7 



3%oV 



V7+ 1 



(16) 



(17) 



for 77700 > 777,4. We find that the solution A a for 7T70o < 
7714 is phcnomenologically unacceptable. 

Numerically, with fixing 7774 = 300 GeV, we find 

A inst = 0.44 (0.44) TeV, 0.71 (0.74) TeV, 1.6 (2.1) TeV, 

(18) 

for 

77700 = 200 GeV, 300 GeV, 400 GeV, (19) 

and 

A A = 4.3 (3.7) TeV, 2.5 (2.4) TeV, 2.1 (2.1) TeV, (20) 



for 



77700 = 500 GeV, 600 GeV, 700 GeV, (21) 



where the values in the parentheses are the full one-loop 
results. The approximation works well. 

In passing, the compositeness conditions [13], l/y\ — > 
and A/y| — > 0, require 77 = and then we get the 
relation between 7774 and m^o ; 



(22) 



Numerically, it gives 

77700 = 480 GeV, 583 GeV, 713 GeV, (23) 
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FIG. 2: M t > v.s. M v (upper left), M T , v.s. M v i (upper right). M t , v.s. M T , (lower left) and M t ; - M v v.s. M r / - M v , 
(lower right). The data points are the same as those in Fig. 1. In the upper figures, the blue dashed lines correspond to 
My^i) = Mt>( T i) ± Mw- The blue lines in the lower right correspond to ±Mw- Notice that the decay channels £'—»&' + W + 
and t' — > v' + W~ are kinematically allowed in the parameter regions M t i — My > Mw and M r i — M u i > Mw, respectively. 
In the lower right, the dashed semicircles correspond to the relation (26) with M^o = 300, 500 GeV. 



for 

m 4 = 300 GcV, 350 GcV, 400 GcV . (24) 

We now proceed to perform the full analysis of the 
one-loop RGE's. 

For a quark q, we read the MS-mass via [24] 

m q {ii = m q ) = M q (l - ^ + 0{a 2 s ^j , (25) 

where M q and rh q denote the pole and MS-masses, re- 
spectively. We used a s (Mz) = 0.118. For leptons and 
the Higgs, the tree level formula is utilized. We var y 
the fermion masses [24], 256 GeV < M v < ^87r/5w, 
255 GeV < M v < y^/5v, 100.8 GcV < M T > < V^v, 



and 90.3 GeV < M u i < \&nv without any prejudice, 
where ^87r/5u ~ 552 GeV and V8^v ~ 1.23 TeV are 
the perturbative unitarity bounds for quarks and lep- 
tons, respectively [17]. We took into account all of the 
40 patterns of the mass spectrum of the fermions 1 and the 
corresponding threshold effects. For the Higgs mass, we 
survey the parameter space, 114 GeV < M^o < \/~Akv (~ 
873 GeV). Imposing the perturbative unitarity bounds 
on all yukawa couplings, and the stability and triviality 
bound on A, < A(/i) < 2-7T, we can estimate the theo- 



Owing to the mass bounds M t i , My > Mt , the possible patterns 
arc reduced into 40 from 5! = 120. 
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retical cutoff scafe A for the SM4. 

We also take into account the constraints from the 
oblique parameters [21]. In order to suppress the S- 
parameter, M t > > My and/or M T i > M v i are fa- 
vorable. Although it increases the T-parameter, this 
is rather nice, because a relatively heavy Higgs pulls 
down T [3, 5]. As for estimate of S and T, we follow 
the LEP EWWG [25]. We obtain the central value as 
(S,T) = (0.06,0.08), where the SM point is normalized 
to (S, T) = (0, 0) and the top mass M t = 173.1 GeV and 
the reference Higgs mass M^o >re f = 117 GeV are used. 
The relevant experimental values in the estimate are 
M z = 91.1875±0.0021 GeV, Aa[^ d = 0.02758±0.00035, 
M w = 80.399±0.0025GeV, sin 2 6» off = 0.23153±0.00016, 
and T e = 83.985 ± 0.086 MeV [26]. 

In Fig. 1, we depict scatter plots M t > v.s. M^o and 
M„> v.s. M o within the 95% C.L. limit of the (S,T)- 
constraint. In each point, the fermion masses are dif- 
ferent. We also showed the (S, T)-contour and the data 
points in the inset of the left of Fig. 1. For consistency 
of the model, the cutoff scale should not be so small. In 
the figure, we took the cutoff A > 2 TeV. 

We find that the theoretical lower bound of the Higgs 
mass is M o > M t >, when A > 2 TeV. (If we take A > 
1 TeV, slightly lower values of M^o are allowed, M^o > 
Alt' — 50 GeV.) Note that the Higgs production via the 
gluon fusion process is considerably enhanced owing to 
the loop effects oft' and b'. For example, the cross section 
v gg ^4>° for M v = My = 0.4 TeV and M r = 0.5 TeV 
increases by a factor of 5. Depending on the masses of 
t',6' and <f> , the enhancement factor varies from 5 to 
9. Consequently, a wider mass range of M^o should be 
excluded at Tevatron. This potentially excluded mass 
range is fairly lower than the above Higgs mass bound, 
however [27]. 

In addition, the right of Fig. 1 clearly shows that 
the decay channel (j>° —> v'v' is opened in a favorable 
parameter space. The importance of this process has 
been emphasized in Ref. [5], i.e., the new signal via 
<f>° —> v'v' — > 4£ + Ifi, where If is the missing energy, can 
be comparable to the rate for <jp — > ZZ — > 4£. Moreover, 
we find that there is a parameter region where <f>° — > t't' 
is also kinematically allowed. In fact, several scenarios 
are possible. We show data samples in Table I. 

The constraints from the oblique parameters cause 
strong correlations between M t > and My and also be- 
tween M T i and M v i , as shown in the upper left and right 
of Fig. 2. The lower left of Fig. 2 suggests that there is no 
correlation between M t < and M T i, as expected. On the 
other hand, the fermion mass differences are strongly cor- 
related, as shown in the lower right of Fig. 2. This essen- 



tially corresponds to the constraint of the T-paramctcr, 
3{M t ,-M b ,) 2 (M T /-Af„,) 2 

M 2 W M^r 

« (1.3-2.0) + 1.41n— — . (26) 
M^o^ re f 

We depicted it with M^o = 300 GeV, 500 GeV and 
M^o re f = 117 GeV in the semicircles of the lower right 
of Fig. 2. The S'-parameter constraint also suggests that 
the parameter region M T i > M v < and My > My is fa- 
vorable. (Within the 95% C.L. limit, M v < My is also 
possible with paying cost of a worse x 2 ; as shown in the 
lower right of Fig. 2. ) 

We emphasize that in a wide parameter region, we find 
M T i > M v i + M w , i.e., the decay channel t' -> v' + W~ 
is allowed. (The situation is unchanged, even if we take 
A > 1 TeV.) Also, t' -+ b' + i s possible. These do 
not necessarily contradict the results in Ref. [5]: Since 
the Higgs is inevitably heavy in our approach, the T- 
parameter constraint requires a larger mass difference of 
the fermions than that of Ref. [5]. (The \ 2 is a bit worse, 
however.) 

The implications of Figs. 1-2 are obvious: If t' and/or 
b' are discovered at the Tevatron and/or LHC, the 
Higgs mass will be suggested under the assumption of 
the SM4. On the other hand, if the LHC excludes 
Mf,y 500 GeV, only few points survive when we take 
A > 2 TeV. I.e., a model with the cutoff A < 2 TeV or a 
nonperturbative regime will be left to be explored. 



III. TWO HIGGS DOUBLET MODEL 
A. Model 

Let us consider the THDM with the fourth generation: 
£thdm = Aon — £y — V, (27) 

where £kin represents the kinetic terms of the fermions, 
the Higgs fields and the SM gauge fields, Cy denotes 
the yukawa couplings between fermions and the Higgs 
fields, and V is the Higgs potential. The yukawa sector, 
in particular the neutrino one, is model-dependent. For 
simplicity, wc assume that the neutrinos have the Dirac 
masses. A model with Majorana neutrinos should be 
considered separately. This is, however, out of scope in 
this paper. 

We define the THDM of the type II (THDM II) with 
the Dirac neutrinos as follows: One Higgs doublet ($i) 
couples to the down-type quarks and charged leptons, 
while the other ( < &2) does to the up-type quarks and neu- 



6 







71 /T 71 yf 


71 /T 71 /T 

M v i M T i 


A 


S T 


(al) 
(a2) 


0.47 
0.59 


0.36 0.34 
0.45 0.42 


0.092 0.23 
0.092 0.24 


3.5 
2.3 


0.20 0.22 
0.21 0.22 


(bl) 
(b2) 


0.45 
0.55 


0.32 0.36 
0.40 0.41 


0.098 0.23 
0.13 0.28 


8.0 
3.4 


0.23 0.27 
0.24 0.22 


(cl) 
(c2) 


0.51 
0.65 


0.44 0.36 
0.50 0.42 


0.10 0.17 
0.097 0.18 


2.1 
2.1 


0.24 0.28 
0.25 0.29 


(dl) 
(d2) 


0.39 
0.55 


0.32 0.34 
0.43 0.41 


0.093 0.23 
0.13 0.28 


2.4 
2.1 


0.21 0.23 
0.24 0.23 


(el) 


0.49 


0.36 0.34 


0.26 0.40 


2.1 


0.26 0.24 



TABLE I: Data samples for several scenarios. The mass unit 
is TeV. For (al), (bl), (cl), (dl) and (el), we took the mass 
bounds M t i > 311 GeV and My > 338 GeV [28], whereas 
we did M t i,v > 400 GeV for (a2), (b2), (c2) and (d2). For 
all samples, we took A > 2 TeV. The criterion for (al) and 
(a2) is the x 2 -ininimum. Similarly, (bl) and (b2) have the 
largest A within the 95% CL. limit of the (S, T)-constraints. 
For (cl) and (c2), t' -> b' + W + is possible. The samples (dl) 
and (d2) are most favorable data for M^o < 2M T i , while (el) 
is for M^o < 2M„;. We do not have the data sample with 
M t , tb > > 400 GeV and M^o < 2M V , . 



tral leptons, i.e., 

4 



+ E Ygt®u%>$ 2 + E y^^^u (28) 

where represents the right-handed up-type quark of 

the j-th family and the definitions of etc. are then 
self-evident. 

The Higgs potential is 



V 



= m 2 $|$i -f 

-A!($l$!) 2 



h A 2 ($J$ 2 ) 



mf 2 (${$2 + (h.c.)) 
' A 3 ($ t 1 $i)($ 2 ^ 2 ) 



+A 4 ($ 1 $ 2 )($ 2 $i) + ^A 5 (($^ 2 )($ t 1 $ 2 ) + (h.c.)),(29) 

where we do not consider the hard Z 2 -breaking terms. 
Owing to the (softly broken) Z 2 -symmetry, the tree-level 
FCNC is absent [29]. 

We do not consider CP violation in the (tree-level) 
Higgs sector and hence will take all parameters in the 
Higgs potential V to be real, so that there are eight pa- 
rameters. In the yukawa sector, we assume that the mix- 
ing terms between the fourth generation and the others 



are absent, i.e., Y(j = Y v 



v4t 



= Y* 4 = y u ,5 4k , and Y 4k = Y| 4 = y T ,5 ik . We can 



then reduce number of parameters. 

When the EWSB occurs, three (G° and G ± ) of the 
eight scalar degrees of freedom are eaten by the weak 
gauge bosons. The physical mass spectrum then contains 
two CP even Higgs bosons h and H defined by Mh < Mh, 
one CP odd Higgs A, and the charged Higgs pair H ± , so 
that the original Higgs fields are written in terms of the 
physical degrees of freedom as follows; 



V2(c p G+ - Sf} H+) 
s a h + c a H + i(cpG° 



s p A) 



and 



(30) 



$2 = 71 



\/2( S/ ^ 
spv + c a h + s a H + i(spG° + cpA) 



(31) 

where a is the mixing angle between h and H and the 
ratio of the VEVs of the two Higgs fields is defined by 
tan /3. We also used the notations sp = sin/3, cp = cos/3, 
and etc.. 

It is convenient to express the quartic couplings A1-5 
through the Higgs masses, the soft Z 2 -breaking term to 2 2 , 
and the mixing angles a and (3: 



2Aiw 2 = 



2AV 



r 2 IM-2 , 2 M 2 

c a M H + s a M h _ m2 ^ (32a) 



A 3 « 2 



X4V 

\ 5 



M tan" 2 0, (32b) 
S -^(M 2 H -M 2 ) + 2M 2 H± -M 2 , (32c) 

S2/3 

M\-2M%± +M 2 , 



X,v 2 



-M% + M 



(32d) 
(32e) 



where we used Al 2 . 



M 



S0C/3 



(33) 



instead of rn\ 2 - Since we consider a general THDM, all 
quartic couplings arc independent and hence free from 
the MSSM relations [10]. 

The decoupling limit of the extra Higgs corresponds to 
M 2 ~ 0{v 2 ) and M 2 H A H± ~ M 2 > v 2 [11], where M 2 
is independent of the quartic couplings A1-5 and thus can 
be taken as some high scale without contradict against 
the perturbative unitarity bound. In this case, sin(/3 — 
a) ~ 1 is also derived. The low energy effective theory 
in this limit is reduced into the SM4. 
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FIG. 3: Scatter plots of M t > v.s. (left) and M v i v.s. Mh (right). The data points are the same in both figures. We took 
sin(/3 - a) = 1 and varied 256 GeV < M t > < 552 GeV, 255 GeV < M b > < 552 GeV, 100.8 GeV < M T > < 1.23 TeV, 90.3 GeV < 
M u , < 1.23 TeV, 114 GeV < M h < 1 TeV, M h < M H < 1.5 TeV, 300 GeV < M H ± < 1 TeV, 93 GeV < M A < 1 TeV, 
0.5 < tan/? < 5, and |As| < it. The red, blue, magenta and green points correspond to the cutoff A, 2 TeV < A < 3 TeV, 
3 TeV < A < 4 TeV, 4 TeV < A < 5 TeV and A > 5 TeV, respectively. All data are within the 95% C.L. limit of the S and T 
parameters. 



0.4 
0.3 
0.2 
0.1 


-0.1 
-0.2 
-0.3 



U=0 




95% CL 
68% CL 



-0.3 



-0.2 -0.1 



0.1 
S 



0.2 



0.3 



0.4 



FIG. 4: The 68% and 95% C.L. constraints on the S and T 
parameters. We also showed the data points in Fig. 3. 



B. Methodology 

Since we assume that all parameters in the Higgs po- 
tential V are real, there are eight parameters in the Higgs 
sector. A convenient choice is to take 

M h , M H , Ma, M h ±, 

A 5 , tan/3, sm{/3 - a), v (= 246 GeV) . 



In this paper, we fix sin(/3 — a) = 1, at which h is SM-like. 
Furthermore, there are four parameters corresponding to 
the pole masses of the fourth family quarks and leptons, 



M v 



My 



M, 



Basically we search a favorable parameter space by vary- 
ing the above ten parameters, as in the analysis of the 
SM4. 

One of the problem is the matching condition between 
the SM4 and the THDM: At least in the decoupling limit 
characterized by M^ - 0(v 2 ) and M\ A H± ~ M 2 > u 2 , 
we need to consider the matching of the two theories, 
SM4 and THDM. We here note that the structure of the 
yukawa sector as well as the Higgs sector is quite different 
in the two theories. In the THDM II, the VEV of the 
Higgs $j (i = 1, 2) provides the fermion mass m, (for the 
fourth generation, 7711,2 = mv ,v or m\ y i = m T iy\ so 
that the relation between the mass rrn and the yukawa 
coupling yi is 



mi 



Vic/3 

7f 



m 2 



V2S/3 

7T 



(34) 



where — yv ,t> or 2/1,2 = y T ',v' for the fourth genera- 
tion. On the other hand, the fermion mass in the SM is 
given by 



(i = l,2). 



(35) 
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Besides, the running effects of the yukawa couplings are 
different. These affect estimate of the cutoff A. 

A simple case is the situation Mh = Ma = M H ± ■ By 
definition, Mh < Mh and thus we can apply the SM4 
up to the scale ^ = Mh = Ma = M H ± , where we will 
identify h to the SM Higgs </>°. Above the heavy Higgs 
scale, fi > M H = M A = M H ±, we utilize the THDM 
description. 

For a general mass spectrum of the Higgs bosons, we 
handle the problem as follows: 

When we randomly generate data of the physi- 
cal Higgs masses M hHAiH ±, we define the light- 
est Higgs mass among them by [i LH , i- e -j ^lh = 
min(M/i, Mh, Ma, M h ±). Similarly, the second lightest 
one is defined by ^ 2 lh- 

When (J. LH = Mh, we can regard the theory in the 
region (i LH < M < M 2 Lff as the one Higgs doublet model. 
(In /i < fx LH , the corresponding theory is "Higgslcss" .) 
We then improve the yukawa and Higgs quartic couplings 
by using the RGE's of the SM4 up to the scale H 2 lh- ^ n 
the region fi > (J. 2 lh> the one Higgs description can't 
be valid. Thus we employ the matching conditions at 

M = M 2 Lff> 

yf M (V = M2Lff) = 2/1 = ^2Lh) C/3 , (36) 
2/2 M (M = ^2LH) = V2 (M = ^lh) s P > (37) 

for the yukawa couplings, and 

M 2 

2A S m(M = ^2LH) = — c 0-a 
V 

+2X 1 s 2 a cj j + 2A 2 c 2 s 2 - iA 3 45S2QS20, (38) 

with A345 = A3 + A4 + A5, for the Higgs quartic couplings, 
where Asm represents the SM one. Eliminating M 2 from 
the THDM relation (32) and using Eq. (38) instead, we 
obtain the quartic couplings A1-5 of the THDM at the 
scale \i = H2LH- Practically, we can find A1-5 by replac- 
ing Ml in (32) by the RG improved SM value. 

On the other hand, if fi LH Mh, the low energy ef- 
fective theory at the TeV scale is no longer the SM4. In 
this case, we may treat the theory as the THDM from 
the beginning. 

In this paper, we do not consider a general case with 
sin(/3 — a) 7^ 1. For a full analysis of the THDM, more 
sophisticated prescription should be required. 



C. Numerical analysis 



We vary the fermion masses, 256 GeV < M t > < 
552 GeV, 255 GeV < My < 552 GeV, 100.8 GeV < 
M T i < 1.23 TeV and 90.3 GeV < M v , < 1.23 TeV. 

For the Higgs sector, we vary the Higgs masses, tan/3 
and A5. 

The Higgs quartic couplings are theoretically con- 
strained by the stability conditions for the Higgs poten- 
tial [15] 2 , 

Ai > 0, A 2 > 0, A 3 > -2y/\iM, (39a) 
A3 + A4- |A 5 | > (39b) 

and also the tree level unitarity bounds [19, 32], 

|a±|,|6±|,|c ± |,|4|,|gi|,|g2|,|/±| J |/i|,|/ 2 | < 16tt£, (40) 
with 

a± = 3(Ai + A 2 ) ± V9(Ai - A 2 ) 2 + (2A 3 + A 4 ) 2 , (41a) 



b± = (Ax + A 2 ) ± ^(Ax - A 2 ) 2 + A 2 , (41b) 

c± = 4 = (Ai + A 2 ) ± yj (Ai - A 2 ) 2 + A 2 , (41c) 

ei = A 3 + 2A 4 -3A 5 , (41d) 

e 2 = A 3 -A 5 , (41e) 

/+ = A 3 + 2A 4 + 3A 5 , (41f) 

/-=A 3 + A 5 , (41g) 

fi = h = A3 + A 4 , (41h) 



where we take £ = 1/2, which corresponds to the radius 
of the Argand diagram. 

Although we will ignore the mixing terms between the 
fourth generation and the others, the charged Higgs mass 
should be severely constrained by b — >• sj and R^, as in 
the three generation model [33]. In this paper, we do not 
fully analyze the experimental constraints. Instead, we 
take M H ± ^ 300 GeV in order to evade the constraint 
from b — >• 57 [34, 35]. We also take into account the 
i?b-constraint for tan/3 < 1 [36, 37]. The B°-B° mixing 
yields less severer constraints only. We do not consider 
too small or too large tan/3, because in such a case, the 
yukawa couplings reach so quickly the Landau pole. 

Eventually, the parameter space for the Higgs sector is 
taken as following; M^ in < M h < 1 TeV, M h < M H < 
1.5 TeV, 300 GeV < M H ± < 1 TeV, 93 GeV < M A < 
1 TeV, 0.5 < tan/3 < 5, and |A 5 | < tt, where Af£ lin corre- 
sponds to the lower bounds of Mh for the various values 



The numerical analysis is similar to that in the previ- 
ous section [30]. 



2 The factors of the quartic couplings Ai and A2 are different from 
those in Refs. [15, 31, 32]. 
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FIG. 5: M t > v.s. My (upper left), M T , v.s. M y , (upper right). M t , v.s. M T , (lower left) and M t > - M v v.s. M T , - M v , (lower 
right). We took sin(/3 — a) = 1. The data points are the same as those in Fig. 3. The blue lines in the upper figures correspond 
to M h i(yt) = M t i( T i\ ± Mw- The blue lines in the lower right correspond to ±Mw- 



of sin 2 {(3 — a) which can be read from the constraints of 
the LEP experiments [38]. In the case of sin(/3 — a) = 1, 
it corresponds to the SM bound, M^ in = 114 GcV. 

Taking into account the RGE's for the yukawa and 
Higgs quartic couplings [31, 39], which are shown in 
Appendix B, and also imposing the instability bounds 
for the Higgs potential and the perturbative unitarity 
bounds on the yukawa and Higgs quartic couplings, we 
calculate the cutoff A at which some new physics or 
strong dynamics enters. The masses of the fermions 
and the Higgs bosons are also constrained by the (S, T)- 
paramcters [3]. 

Since the parameter space is enormous, we need to find 
efficiently the cutoff A unlike in the analysis of the SM4. 
After generating the primary data with an equal proba- 
bility, we refine all parameters so as to make A larger. We 
cannot deny the possibility that we may overlook some 



favorable parameter region, if the primary data might be 
too rough. 

We depict the results in Figs. 3-7. 

The relation between M t i^ and Mh is shown in Fig. 3. 
All data points are within 95% C.L. limit of the (S,T)- 
parameters. (See Fig. 4.) Similarly, the mass relations 
between the fourth generation fermions are depicted in 
Fig. 5. The masses of the extra heavy Higgs bosons are 
described in Fig. 6. The allowed parameter region for 
M H ± and tan/3 is shown in the left of Fig. 7. The val- 
ues of A5 can be read from the right of Fig. 7 by using 
Eq. (32). 

It is noticeable that the decay channels h v'v 1 and 
t' — > v 1 + W~ are allowed in a wide parameter range. 
(See the right of Fig. 3 and the lower right of Fig. 5.) 
Notice that tan /3 ~ 1 in order to relax the appearance of 
the Landau pole for the yukawa couplings. (See the left 
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6: Masses of the extra Higgs bosons. We took sin(/3 
1. The data points are the same as those in Fig. 3. 



of Fig. 7.) 

Schematically speaking, as shown in the left of Fig. 3 
and Fig. 6, the results consist of high and low Mh regimes, 
Mh > M t r and Mh < M t /, respectively, where we took 
the cutoff A > 2 TcV. 

We can confirm that the decoupling regime of the extra 
Higgs bosons, say, M H ~ M H ± > 800 GeV and M A > 
700 GeV, is contained in the region Mh > My. (See 
Fig. 6 and the right of Fig. 7.) This is consistent with 
the analysis of the SM4. In this case, the extra heavy 
Higgs bosons can decay into a quark/lcpton pair of the 
fourth generation, if kinematically allowed. 

A new feature of the two Higgs extension is thus char- 
acterized by the low M h regime, M h < M v ■ In the (S, T)- 
analysis, this regime is more favorable than the high Mh 
one, i.e., most of the data points inside the 68% C.L. limit 
of the (S, T)-constraints are for the former. The point is 
that even for the low Mh, the Higgs potential can be 
stable owing to the dynamics of the Higgs quartic cou- 
plings. When we take into account the Tevatron bounds 
of the fourth generation quark masses, M t > > 311 GeV 
and Mv > 338 GeV [28], only a small parameter space 
is left, however. Nevertheless, we here mention that the 
parameter region with Mh ~ 100-300 GeV is interesting, 
because the extra Higgs masses can be almost degenerate, 
Mh ~ Mjf± ~ Ma ~ 300-400 GeV, and also a scenario 
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with M 2 = is possible (see the right of Fig. 7). We also 
note that even in this regime, the leptonic decays of the 
Higgs bosons such as h — > p'u', H~ — > v'r' and etc. are 
open in a certain parameter space. 

We have analyzed only the case of sin(/3 — a) = 1. If 
we extend our analysis with a general sin(/3 — a), more 
favorable and exotic Higgs mass spectra can be found. 
This will be performed elsewhere. 



IV. SUMMARY AND DISCUSSIONS 

We have reanalyzed the constraints on the mass spec- 
trum of the fourth generation fermions and the Higgs 
bosons for the SM4 and the THDM II with sin(/3-a) = 1. 
We showed that there are the noticeable correlations 
among the mass spectrum of the fermions and the Higgs 
bosons. 

For the SM4, the favorable mass range of the physical 
Higgs boson 0° is M^ > M v (M o > M t , - 50 GeV) for 
the cutoff A > 2 TcV (A > 1 TcV). We also found that 
the favorable parameter space is mainly contained in the 
region, 



3(M t ,-M b ,) 2 (M T ,-M v ,f 



M 2 W 



M 2 



(1.3-2.0) + 1.4 In 



Mrh 



M, 



0° 7 ref 



and 



M V >M V , M T ,>M V ,. 



(42) 



(43) 



(See also the semicircles in lower right of Fig. 2.) We 
showed the data samples corresponding to several sce- 
narios in Table I. 

For the THDM II with sin(/3 - a) = 1, schematically 
speaking, there are two domains for the favorable mass 
range of the light CP even Higgs h, Mh > Mt> and 
Mh ^5 Mf. The extra heavy Higgs decoupling regime 
is contained in the former. This is consistent with the 
analysis of the SM4. On the other hand, an almost degen- 
erate Higgs mass spectrum such as Mh ~ 100-300 GeV 
and M H ~ M H ± ~ M A ~ 300-400 GeV is allowed in the 
latter and in a part of the former. In this model 
with M 2 = is not excluded. (See the right of Fig. 7.) 
As for the value of tan j3, we found that tan /3 « 1 is 
favorable in the both domains. 

Concerning the decay channels of the charged leptons, 
we found that r' — > v' + W~ is allowed in a wide param- 
eter space in the SM4 and the THDM II. (See the lower 
right of Figs. 2 and 5.) The fourth generation quark 
masses can be degenerate M# = Mv or the decay chan- 
nel t' -> b' + W w can be open, depending on the mass 
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tan p 



FIG. 7: tan /3 v.s. M H ± and M 2 /v 2 v.s. M\/v 2 . We took sin(/3 — a) = 1. The data points are the same as those in Fig. 3. We 
can read the value of As from the right figure by using Eq. (32). 



difference M T i — M v i. (See the lower right of Figs. 2 
and 5.) The Higgs 0° in the SM4 and the light CP even 
Higgs h in the THDM II can decay into a pair of the 
fourth generation neutrinos. (See the right of Figs. 1 and 
3.) Furthermore, in the THDM II, a scenario that all 
Higgs bosons decay into a pair of the fourth generation 
lcptons is possible, i.e., h — > v'v' and H~ — > v't' , etc.. 
(For studies of collider signals of the fourth generation 
lcptons, see, e.g., Ref. [40].) We also comment that the 
main decay channel of the heavy CP even Higgs H can 
be H -> Pt',b'b', if kinematically allowed [41]. Thus the 
phenomenology of the fourth generation models is very 
rich. 

The implications of the analysis in this paper are ob- 
vious: If the Tevatron and/or LHC discover t' and/or b', 
the Higgs mass spectrum will be suggested, depending on 
the models. On the other hand, if the LHC excludes the 
t' and/or b' masses M t ^ b i < 500 GeV at early stage, a big 
parameter space will be gone. In this case, essentially, a 
nonperturbative regime will be left to be examined. 

Many issues remain to be explored: 

• We did not consider a general case of sin(/3 — a). 
There probably exist more favorable and exotic pa- 
rameter regions in the THDM II. Moreover, we may 
consider a different yukawa structure other than the 
type II [42]. 

• Majorana neutrinos can reduce S and T [43]. It 
may affect the mass spectrum of the fourth gener- 
ation fermions and the Higgs bosons. 

• The two-loop effects are probably relevant for more 
precise predictions of the mass spectrum. The the- 



oretical lower bound for the Higgs mass, which es- 
sentially corresponds to the instability bound of the 
Higgs potential, will be almost unchanged, how- 
ever, because the parameters certainly stay in a 
perturbative region. 

• We did not take into account the mixing angle be- 
tween the fourth and third generations. This is, 
of course, very important to discuss the flavor con- 
straints and realistic decay chains of the fourth gen- 
eration quarks and leptons [44, 45]. 

• If the main branching ratios of t' and b' are different 
from well-studied ones in experiments, a first evi- 
dence of the fourth generation might be found in 
the Higgs physics, for example, as a huge enhance- 
ment of the golden mode, gg — > (jP/h — > ZZ. Con- 
cerning the loop induced processes, it is important 
to notice that the loop effects in h — > gg, h — > 77 
and A — > 53,77 are quite different [10]. Related 
to such possibilities, there should exist very large 
non-decoupling effects in the triple Higgs coupling 
arising from the fourth generation quarks and lep- 
tons [32, 46]. The triple Higgs coupling is testable 
at the LHC/vLHC/sLHC [47] and at the ILC [48]. 

• The fourth generation model may play an impor- 
tant role in B-CP asymmetries [49, 50] and also in 
the electroweak baryogenesis [51]. 

• Last but not least, if only a nonperturbative regime 
is left in the near future, what kind of study will be 
relevant? For example, when the fourth generation 
quarks are ultraheavy beyond the perturbative uni- 
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tarity bound, is there some nonperturbativc effect 
in the nonresonant gg — > ZZ process [52]? 
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Appendix A: RGE's for the SM4 



For the gauge couplings, the RGE's are 



(16tt 2 ) M |- 5 . 



-Ci9i, 



with 



Cl 
C2 
C3 



-\ NH 

6 

22 4 



20 



N„ 



-N 



H- 



(Al) 

(A2) 
(A3) 
(A4) 



where N g and Nh denote the number of generations and 
the number of Higgs doublets, respectively. Although we 
did not show explicitly the formulae, we take into account 
the threshold effects. 

The RGE's of the yukawa couplings are [22, 23] 



d 

(I6n 2 )n—y t 



9o ^-^2\ S 

4<? 2 + -^9i)Vt + 



+yt 



o 2 , o 2 , 3 2 2, 2, 2 

3y t , + 3y b , + -y b + y v , + y T , + y T 



(A5a) 



+Vb 



m + 3y? + -y\ + yl 



(A5b) 



d 

(167T 2 )/i — ^ 



1 



\V V > 



3y 2 + 3y 2 + 3y t 2 + 3y 2 - -y^, + y^ 



(iG^)^yr' = -(-^gl + ^gDvr' + ^yl, 



,(A5f) 



3j/ 2 + 3y 2 , + 3y. 2 + 3y 2 - iy 2 , + y 2 



(A5g) 



where we did not show explicitly the threshold effects 
of the fermions. Inside of the square bracket, y 2 (/ = 

t', &',-") should be regarded as y 2 0(^ — rrif), where wif 
is the corresponding MS-mass of the fermions. 

The RGE for the Higgs quartic coupling is given by 

(16tt 2 )/^A = 24A 2 - 3A(3.g 2 + g\) 



-4A 

-2 
3 



%2 , 2, 2, 2\ , 2, 2, 2 
t' + VV + Vt + Vb ) + Vt> + Vu> + Vr 

2g i 2 + (g 2 2 +g 2 ) 2 } ■ (A6) 



Appendix B: RGE's for the THDM II 

Let us consider the RGE's for the THDM II. 

The RGE's for Yukawa couplings are given by [31, 39] 

(I67r 2 )/^y t = -(8. 9 2 + \g\ + ^gf)y t + \yl 



+yt 



o 2 i 1 2 i 2 

3y*' + -jVb + <v 



(Bla) 



d 

(16tt 2 )m^2/t 



,9 2 15 2n 

M ff2 + T ffl ^ T 



2^ 



3%< + 3y b , + 3y t + 3y b + y v , + y T , 



,(A5c) 





(167r 2 )Ai^y b 



-(8 53 2 + 752 2 



5 2l 9 3 



-2ft 



3y 2 , 



x 2 i 2 i 2 
2% + 2/t' + Vt 



, (Bib) 



(167r 2 )/z^y t 



d_ 

3 2 



9 9 2 17 2 , 9 3 

75 + 4.92 + ^g\)yt> + 2%' 



3y t 2 + 3y 2 + y 2 , + y 2 , + y 2 



(A5d) 



9 

(16tt 2 )ai— y T 



/9 2 15 2 , 5 3 

-(452 + ^5 2 )yr+ 2?/r 



+2/r 



3y 6 + 3y fc 



y 2 > 



(Blc) 
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-Vt> 



o 2 i 1 2 i 2 

3y t + + V 



(Bid) 



(16tt 2 ) M = -(853 + |?2 2 + ]|<? 2 )2/&' + 



-2/6' 



3y 2 + \vl + yl + y 2 r 



(Ble) 



d_ 



(167r 2 )/i^A 3 = 2(Ai + A 2 )(6A 3 + 2A 4 ) + 4A 2 + 2A 2 



-2A 2 -3A 3 (3. 9 2 + .g 2 ) + - 



252 + (5 2 2 - alf 



+2A 3 
-4 



3(y t 2 + yl + yl + vl ) + yl + yl + yl 



"Zylyl + SytVb + vl'vl' 



(B4) 



-yv 



3y 2 + 3y 2 , + -y 2 T , 



(Blf) 



/.„ 9n d ,9 , 15 \ 5 , 

(167r 2 )ti—y Tl = -{-gl + —g{)y T ' + -y\, 



-fir 



3y 2 + iyl + \yl + yl 



(Big) 



where we ignored y c , y v , etc.. 

The RGE's for the Higgs quartic self-couplings are [31, 
39] 

(16vr 2 )/i^-Ai = 24A 2 + 2A 2 + 2A 3 A 4 + A 2 + A 2 
on 



-3\ 1 (3g 2 + g 2 ) + - 



2ff2 + (5 2 2 +3 2 ) 5 



+4Ai iy 2 b , + 3y 2 + y 2 T , + y 2 T 



3yt, + iyt + y\, + y\ 



(B2) 



(16tt 2 ) m ^-A 2 = 24A 2 + 2A 2 + 2A 3 A 4 + A 2 + A 2 



-3A 2 (3.g 2 + ff 2 ) + 
+4A 2 3y 2 t , + 3y 2 + yl, 



3yf, + 3yf + yt> 



2<? 2 4 + (ff 2 2 +S 2 ) 5 



(B3) 



(167t 2 )m^-A 4 = 4(Ai + A 2 + 2A 3 + A 4 )A 4 + 8A 2 , 



-3\ A (3g 2 +g 2 ) + 3g 2 g 2 
-2A 4 



+4 



3(y 2 + yl + yl + yl) + yl + yl + yl 

3y 2 y 2 +3y 2 y 2 + yl iy 2 T , , (B5) 



{i&iT 2 )n— A 5 = a 



dji 



4(Ai+A 2 ) + 8A 3 + 12A 4 
-3(3.g| + g{) + 2 j 3(y 2 + y 2 + y 2 + y 2 ) 
+yl + y 2 T , +yl}}. (B6) 



Note that the definitions for Ai and A 2 are twice larger 
than those in Ref. [31]. 
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